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(57) Abstract 

A virtual microscope slide is created by using a computer-controlled microscope (10) to capture a plurality of low maenification 
-mages of a specimen which are tiled to create a reconstructed macro image (24). A plurality of higher magnificat 
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METHOD AND APPARATUS FOR CREATING 
A VIRTUAL MICROSCOPE SLIDE 

Field of th e Invention 
This is a Continuation- In-Part of prior patent 
application Serial No. 08/805,856, filed March 3, 1997, 
which is hereby incorporated by reference in its 
entirety. This invention relates to a method of , and an 
apparatus for, acquiring and constructing tiled digital 
images from a specimen on a support, such as a microscope 
slide, and for storing, and transferring the image for 
viewing by another at a local or remote location. 



Background of the Invention 
The invention described in the aforesaid 
application answers a need, a requirement to image and 
15 digitally record an object in a relatively flat plane at 
high resolution/magnification. Today, it is impractical 
to construct an optical image sensor large enough to 
cover the entire image area e.g., of a specimen on a 
microscope slide, at the required resolution. This is 
20 because lens size and resolution/magnification issues 

limit the size of the field of view of magnified objects 
and their resulting images. Viewing through a microscope 
is akin to viewing through a periscope in that , one sees a 
very small field of view even at low magnifications, such 
as 1.25X. A pathologist using a microscope often scans a 
slide to obtain in his mind an overall view or sense of 
what constitutes the specimen and he remembers the 
general locations of the diagnostically significant, 
small pieces of the specimen. Usually, these are the 
diseased areas, such as malignant or potentially 
malignant portions of the specimen. To obtain higher 
resolution and magnification of these suspicious 
portions, the pathologist switches to a higher 
magnification objective lens but then the field of view 
15 becomes much smaller again. Often, the pathologist 
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switches back and forth between the lower magnification, 
larger field of view objective lens to orient himself 
relative to the specimen and the high magnification, 
smaller field of view to obtain the detailed, high 
resolution view of the suspicious area on the specimen. 
Thus, the user never receives a magnified, condensed 
overall view of the specimen or a portion of the specimen 
but must remember the series of views taxen at low 
magnification. Likewise, at high resolution, high 
magnification, the user never receives or views a 
collection of adjacent images but must interrelate these 
successive images in the user's mind. 

A similar problem exists on the Internet or 
intranet where a pathologist may receive a single field 
15 of view magnified image taken from a specimen over the 

Internet or the intranet on his browser. The pathologist 
must be provided with explanations to coordinate the high 
resolution view with the lower resolution view. The 
number of views available to the pathologist is very 
2 0 limited, and the pathologist is unable to select other 

views or to scroll to neighboring views at the areas that 
are most interesting to the pathologist . 

In the aforesaid prior application, there is 
disclosed a method and apparatus whereby a person may 
25 construct a low magnification, digitized overall, image 
view of the entire specimen on a slide or a selected 
portion of the specimen on a slide, such as the basal 
layer of a tissue section. The overall, low 
magnification digitized image allows the user to 
30 understand where the user is presently located in his 
viewing and where the user may want to make the next 
observation. That is, the low magnification overall view 
is generally in color and provides to the experienced 
user a visual overall or thumbnail view of the slide and 
35 shows the possible areas of interest for malignancy or 
other diseases which manifest themselves at certain 
locations on the specimen image being viewed. This low 
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magnification overall view enables the user to select 
thereon the points of interest that the user wants to 
view at a higher magnification. 

The overall view was constructed by taking by a 
5 large number of low magnification images of the specimen 
through a microscopic scanning system and then coherently 
assembling and coordinating these respective smaller 
views or images (hereinafter "image tiles") into one 
coherent, low magnification, macro image from the 
10 specimen. Often the digitized macro image is reduced in 
size by a software system to even a smaller size to be 
displayed on a local screen or to be transferred over a 
low bandwidth or a high bandwidth channel to a remote 
viewing screen . 

15 The prior application teaches how to assemble a 

large number of image tiles, for example, 3 5 image tiles 
for the macro image, and then to take a series of other 
tiles of a higher magnification or magnifications which 
will also be viewed by the user. To this end, the user 
2 0 is provided with a marker, such as a cursor or the like, 
to select the defined area of interest, and by a simple 
command, to cause the selected, higher magnification 
digitized images to appear on the screen for viewing by 
the user. The higher magnification images may be one of 
25 several magnifications or resolutions such as 10X, 20X 
and 4 OX. 

As disclosed in the aforesaid application, it is 
preferred to allow the user, such as a pathologist, to 
quickly flip back and forth between the high resolution 

30 micro image and the low macro resolution image or to 

provide separate split screens whereby the pathologist is 
shown an overall macro view and a marker showing where 
the current higher magnification view is located. 
Because of the multiple magnifications, the user may 

35 change to an intermediate magnification such as would be 
accomplished by switching between intermediate objective 
lenses. This provides the pathologist with views which 
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correspond to changing back and forth between objeccive 
lenses in a microscope, a procedure with which most 
pathologists are familiar and have been trained. 

Additionally, the aforesaid application provides 
5 the user with a scrolling feature that allows the user to 
shift into the viewing screen adjacent, magnified images 
on the screen so that the pathologist is not limited to 
only seeing just a full tile view but may see adjacent 
image material from adjacent, neighboring tile images. 
10 in the aforesaid patent application, there is a 

disclosure of transmitting the low magnification image 
over a local area network or over the Internet through 
various servers and computers. The tiled images that 
were being transmitted were achieved by use of a fully 
15 computer controlled microscope which allowed the user to 
navigate along a specimen area of interest, such as along 
a basal area or to other suspicious points spread 
throughout the specimen to acquire tiled images of 
selected areas so that the entire specimen would not have 
20 to be digitized and stored. As disclosed in the 

preferred embodiment in the aforesaid application, an 
Internet browser remotely-controlled, automated 
microscope could be used by a pathologist from a remote 
location to view the reconstructed macro image tiles; 
25 and, with his manipulation of the microscope, using an 

intranet or Internet browser, could acquire single images 
at higher magnifications if desired. While several 
people could see the particular digitized images being 
transmitted out over the Internet as they were acquired 
30 by a particular pathologist and several people could view 
the stored images, there was still a problem of control 
at operation of the microscope by each person viewing the 
digitized images, and a problem with acquiring and 
transmitting large areas of higher magnification images 
35 using the tiling method. 

As stated above in greater detail, the current 
state of archiving the digital images achieved through a 
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microscope is often by having photographs or by video 
tapes. The photographs are difficult to use as is a 
video tape particularly when the user wants to move 
rapidly back and forth between various images and to 
5 scroll through various adjacent parts of the specimen 

image. Further, current archival methods lack an overall 
macro image of the specimen, which allows the user to 
know exactly where the particular high resolution view is 
from when it is making an analysis of the high resolution 
10 image . 

While digitized images can be stored 
magnetically or otherwise digitized and recorded on 
various recording mediums, no current archival system 
allows the user to toggle back and forth between high 
15 magnification images and low magnification images or • 

between various images at different magnifications such 

i as that achieved by a pathologist switching microscope 

objective lenses in real time to get the macro and micro 
images from the same location on the specimen. 

;: 20 Heretofore, the practice of pathology has been relatively 
limited to the use of microscopes and to the pathologist 
having to use the microscope to review the particular 
specimen . 

There is a need for a dynamic system whereby one 
25 or more or several pathologists, including a consulting 
pathologist, may view the same area simultaneously and 
interact with one another either in diagnosis or in 
analysis. Also, it would be best if the images from the 
specimen could be stored so that a pathologist could 
30 easily examine the images at his leisure using an 

intranet or Internet browser at a later date merely by 
accessing the particular web site where the images are 
located. 

It will be appreciated that a host of problems 
35 need to be solved to allow Internet or intranet users to 
view on their respective monitors useful, low resolution, 
macro images and high resolution, micro images of several 
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adjacent, original microscope images. One of the first 
problems is how to seam together neighboring tile images 
to form a seamless overall view of these tiles. 
Heretofore, attempts to seam the tiles used software to 
combine the pixels at the tile boundaries and have been 
generally unsuccessful. Another problem is that of 
mapping of coordinates beginning with the coordinates, 
usually X and Y coordinates, from and at the microscope 
stage carrying the slide and then the mapping of 
coordinates on the scanning screen not only for one 
magnification but also to coordinate the mapping for the 
respective multiple resolution images taken typically at 
1.25X, 10X and 4 OX or more. These coordinates must be 
maintained for a large number of tiled images, e.g., 40 
tiled images for one macro image. In order for the 
remote user to view these tile images and to flip back 
and forth between different resolution, tiled images, the 
user's computer and monitor not only must receive the 
addresses and stored parameters for each pixel but must 
also run them on a generic viewing program. 

Another problem with acquiring image tiles and 
sending them over a low bandwidth Internet channel is 
that both the storage requirements on the server and the 
amount of data acquired per slide become high, such as 
25 for example, 120 megabytes to one gigabyte. The 120 

megabytes is only achieved by not taking image tiles of 
the entire specimen but only image tiles from the areas 
selected by the pathologist when tracing at high 
resolution along basal layers or only at the dispersed, 
3 0 suspicious cancer appearing area in a breast cancer. 

Even with this selective interaction by a pathologist in 
constructing the macro and micro digitized images with a 
vastly reduced amount of image tiles relative to that 
which would be acquired if the entire specimen where 
3 5 imaged at each of the multiple magnifications, the 
acquired amount of data is a monstrous problem of 
transmitting in a reasonable amount of time over a narrow 
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bandwidth channel to an ordinary web browser having 
limited storage capacity. While rough compression 
techniques could be used, they cannot be used at the 
expense of providing the high resolution image that the 
pathologist must have for diagnosis of the specimen. 

Summar y of the Invention 
In accordance with the present invention, there 
is provided a new and improved method and apparatus for 
constructing digitally scanned images from a microscope 
specimen, for storing the digitally scanned images in a 
tiled format convenient for viewing without a microscope, 
and for transferring the tiled images at multiple 
magnifications for viewing by another at a remote 
location. This is achieved by assembling together 
15 several adjacent, original microscope views at a first 
magnification to obtain an overall macro view of the 
specimen and assembling together several adjacent 
v original microscope views at a higher magnification to 
^ create a combined data structure. The data structure may 
20 then be transferred to the remote viewer to provide this 
viewer multiple resolution macro and micro images of 
areas on the slide specimen. The data structure is 
constructed by digitally scanning and storing the low 
magnification images with their mapping coordinates and 
25 likewise, digitally scanning and storing higher 

magnification images with their mapping coordinates. 
Further, a pathologist may interactively select only 
those diagnostically significant areas of the specimen 
for digital scanning and storing to reduce significantly 
30 the number of image pixels stored at high resolution. 

The data structure can be transmitted over the 
Internet or intranet to allow multiple users to consult 
on a particular microscope each using his own virtual 
images of the specimen. These users each may flip back 
35 and forth between different resolution images in a manner 
similar to that achieved when shifting among objective 
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lens for different resolution views. However, the 
preferred embodiment of this invention provides a marker 
on the overall macro view showing the remote user where 
the higher resolution image is located on the specimen so 
that the user does not have to remember the location of 
the higher resolution image. Unlike the single, small 
optical field of view currently available, the remote 
user is provided with a series of abutted, tiled i-ges 
each being substantially equal to one small optical field 
of view. Thus, the remote user is provided with better 
and larger macro and micro tiled images than the single, 
small optical fields of view taken at the same 
magnifications of a single tiled image. 

The preferred data structure is also provided 
15 with a generic viewing program that allows the -mote 

user to manipulate and interpret the tiled images on the 
user's browser. This generic viewing program is self- 
contained with its own display and the interpretative 
program is usable with a variety of computer,, browsers 
and monitors. The data structure uses selectively 
compressed data to reduce the huge amount of acquired 
data e.g. 120 megabytes, into a small amount of data, 
eg 1.4 megabytes. Such smaller, more manageable 
amounts of data can be transmitted over a low bandwidth 
channel such as the Internet without the loss of 
resolution that would interfere with the remote 
pathologist's analysis. Further, the interactive program 
allows the pathologist to scroll and to view neighboring 
image areas of neighboring image tiles which were 
30 currently unavailable to the pathologist until the 

inventions set forth in the aforesaid application and in 

this application. 

Turning now in greater detail to aspects of this 
invention, problems with achieving tileable Ci.e^ 
35 contiguous images which can be seamlessly abutted next to 
each other to recreate the original image, but at 
different magnifications) multiple images of a specimen 
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on a microscope slide are overcome by the system of the 
invention. The system includes a microscope and stage in 
which digital locations on the stage have been 
predetermined in accordance with an electromechanical 
5 addressable coordinate system <X-Y for convenience) . 
Each point on the stage is assigned an "X" and a "Y" 
coordinate which uniquely defines its location. The 
increments in each of the X and Y directions are 
established at a predefined amount for example in .1 
10 micrometer increments. A key factor in achieving 

superior resolution of the specimen images at higher 
magnifications is to establish many more physical 
increments on the stage for each pixel of the image 
sensor and of the intended display. For example, at 
15 1.25X magnification, 64 points on the stage correspond to 
one pixel on a CCD optical sensor, which corresponds to 
one pixel on a 640 by 480 monitor (for a VGA display), 
using the bitmap addressing and scrollable image method 
described herein. 
20 Once the coordinate system is defined for the 

microscope stage, when a specimen on a microscope slide 
is placed on it, each feature of interest on the slide 
can be uniquely located with reference to the stage. 
Then the microscope system is used to digitally scan the 
25 image. The first scan is done at a relatively small 

magnification since this image will be used to provide a 
"macro" image of the entire specimen. In the preferred 
embodiment, 1.25X magnification is used. The microscope 
system then scans the slide using the 1.25X objective. 
30 Since the image is detected by rectangular optical 

sensors, such as the optical sensors in a CCD grid, the 
stage must be moved in relatively larger increments to 
place the next adjacent physical part of the slide 
exactly in the region where that rectangular area will be 
35 precisely imaged on the CCD sensor. 

Although the area traveled is relatively large, 
the precision must be high to enable alignment of the 
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Since the nature of optics, i.e. lenses, is that 
they provide a generally circular image with a sharp 
central region and with fuzziness around the periphery of 
the image, the microscope system is designed to step 
5 through the various locations on the slide in such a 

manner to scan only the high resolution image portion in 
the center of the optical image. The fuzzy outer regions 
are discarded. This also has the benefit of ensuring a 
high resolution image once the tiled images are 
10 reconstructed for viewing by a user on a monitor. 

After the macro image is completed, a trained 
professional, such as an examining pathologist, views the 
image of the specimen by viewing the macro image and 
looking for areas of interest. In general, most specimen 
15 slides contain only a few small areas of diagnostic 
significance. The balance of the slide is generally 
empty or not significant. When the examining pathologist 
views the slide, some areas may have been previously 
marked in the regions of interest for viewing and 
i20 analysis at higher magnifications. Once these regions 
are marked, the microscope is set to the desired higher 
magnification and then only the marked regions are 
scanned and stored. Alternatively, he may define new 
areas directly on the macro image. In either case, the 
25 regions are outlined using a pointing device, such as a 
mouse, directly on the viewing window displaying the 
macro image. As described above with respect to the 
1.25X images, since the stage has a predefined coordinate 
system, the scanned higher magnification image portions 
30 can be easily located with respect to the macro image, 
creating a series of micro images. 

The fact that a typical microscope specimen 
slide contains only limited information of interest and 
the ability of the system embodying the invention to 
35 accurately locate such regions enables the system to 

create a virtual microscope slide, i.e. a data structure 
which can be used in place of the actual specimen slides. 
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This advantageously enables multiple users to consult on 
a particular specimen. Additionally, because of the 
reduced size of the data structures, they can be viewed 
locally on a personal computer, transmitted over an 
intranet or via the Internet globally. The created data 
structures can be stored on a variety of storage or 
recording media: for example, on a server's hard disk, a 
Jazz drive, a CD-Rom or the like. Storing the data 
structure on a portable storage media further enables the 
transfer and archiving of the microscopic slide data 
structures by multiple users. 

Another feature of the invention is a self- 
executing data structure. This is achieved by packaging 
the tiled images with an active, dynamic control program. 
IS When an active dynamic control program is used by a 

viewing program such as a common web browser, the browser 
can interpret the dynamic control program. This allows 
the user to interact and control the viewed images seen 
on the viewer's screen from the recording medium. More 
specifically, in the preferred embodiment of the 
invention, a large number of low magnification, digitized 
tiled images are formed and embedded in a data structure 
with linking information allowing them to be coherently 
tiled to each other during viewing to form a macro image, 
and a series of higher magnification tiled images also 
similarly constructed into a micro image, and a control 
program such as a JAVA applet, is provided and 
transferred with the macro and micro tiled images for use 
by a remote user. Thus, for example, the macro and mxcro 
tiled images with their active control program may be 
transmitted over an Internet or an intranet to a browser, 
or other application program for viewing the rmages, 
where the user may then access the browser to analyze the 
images at multiple resolutions and with a macro field of 
view before the user. This enables the viewing of the 
images in a manner similar to the use of an optical 
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microscope, but in this case visually the view is of a 
virtual microscope slide at multiple resolutions. 

Also, in accordance with the invention, the 
constructed, tiled macro and tiled micro images along 
5 with the control program can be placed on a web server 
and can be accessed locally and over a wide area, even 
globally, by multiple users at various times. For 
instance, a large number of previously scanned and 
recorded specimen slides, such as 300 specimen slides, 
10 may have their respective micro and macro tiled images 

put on a server. Medical students or pathology students 
then may each access the slide or all of the 300 slides 
and review them on their respective web browsers at their 
leisure. Likewise, a pathologist may dial up or 
15 otherwise connect through an internet service provider to 
the Internet or other long-level network and access a web 
> server and obtain a particular patient's specimen 
v results. Those results would have been stored as a data 
structure (including macro and micro tiled images along 
20 with the control and interpretative program) . The 

pathologist then may and perform an analysis at his home 
or in his office without needing to have or to control a 
microscope or the particular slide. The pathologist may 
toggle back and forth between the micro and macro images, 
25 and then dictate or otherwise prepare his analysis, 

findings or diagnosis from these stored images. This 
advantageously enables the pathologist to perform part of 
his job in the convenience of his home or office and also 
enables a laboratory to maintain actual specimen slides 
3 0 in a safe and secure location, away from the potential of 
damage and without the necessity of shipping the slides 
for microscopic examination at a remote location. 

The control program, which in the preferred 
embodiment of the invention is a dynamic self -executing 
3 5 program such as a JAVA applet, allows the user to 

manipulate and interpret the images while on a browser. 
The dynamic, self -executing program is completely self- 
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contained with its own display and interpretative progra 
for operation by the user of the browser. 

The present invention is not limited to use on a 
browser since the tiled, digitized images and the active, 
control program may be stored on a CD-ROM or other 
portable storage medium and sent through the mail, or 
otherwise transferred to the user for review at the 
user's convenience with dedicated viewers. 

Thus, from the foregoing, it will be seen that 
there is provided a new and improved method of and 
apparatus for archiving of microscopic slide information 
on a storage medium with an active control program, which 
allows the display and interpretation of various micro 

and macro images . 
1S i n accordance with another important aspect of 

the invention, there is provided with the sel f - execut ing 
data structure (the stored macro images, micro images and 
dynamic, sel f -executing program for viewing, 
reconstructing and manipulating the stored images) the 
20 ability to scroll through the displayed images. This 
allows the user not only to see one image tile at a 
particular magnification, but also to use a pointer or to 
otherwise move a point to cause displayed images from 
adjacent neighboring image tiles which were not 
25 previously viewable to be included in the field being 
viewed by the user. That is, the user may shift the 
viewing location across tile boundaries from one tile to 
another, and up or down, or right or left or to other 
points of interest in a normal two-dimensional scrolling 
3 0 manner. Thus, the user is provided with an archived 
stored slide at multiple magnifications which can be 
readily scrolled through in any arbitrarily chosen 
direction or directions. As in the aforesaid 
application, the user interactively will go to various 
35 areas of selected interest and operate a pointer or a 
marker to select for high magnification viewing the 
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particular area of interest and also do a scrolling of 
neighboring areas of interest. 

In addition to the Internet browser, the data 
images can be viewed, reconstructed and manipulated using 
5 a dynamic, self -executing program such as, for example, a 
JAVA applet or an ACTIVE-X applet. An advantage of using 
a dynamic, sel f -executing program which is linked with 
the data images on a data structure is that the data 
images can be viewed, reconstructed and manipulated 
10 independent of the operating system of the users 

computer. Additionally, the user does not have to 
acquire the latest version of the dynamic, self - executing 
program since it is already linked with and provided with 
the data images on the data structure or on the storage 
15 medium. Thus, the user can always view the data images, 
regardless of different program versions. 

Tne dynamic, self -executing program permits 
interchanging the image in its entirety simulating the 
visual effect of changing objectives in a regular, 
20 mechanical optical microscope view. Thus, the user can 
easily switch from one magnification to another and 
scroll through portions of the image, simulating tracking 
the image by moving the slide under the microscope lens. 

The dynamic, self -executing program permits 
25 scrolling the image in a window to enable viewing of the 
reconstructed large field of view images. The user can 
use a mouse, or other pointing device, to select a 
portion of the image on the large field of view image and 
the program will display that selected portion in another 
30 window at the desired magnification. 

A method of constructing a record of the digital 
image of a specimen on a microscope slide using image 
tiles includes scanning the image at a first low 
magnification so that substantially all of the specimen 
35 is obtained. Then the specimen is scanned at a second 

higher magnification so that images of selected (or all) 
sub-portions of the specimen are obtained. The spatial 
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relationships of the first lower magnification image to 
the second higher magnification images is used to 
reconstruct the image during viewing. The individual, 
sub -portions or tiles of the scanned image are seamed 
S together by the dynamic, self -executing program to create 
a digital image of substantially larger areas than 
individually acquired image fields of view without 

tilin9 ' A data structure according to the invention is 
10 created by first digitally scanning the desired specimen 
at a plurality of image magnifications. The scanned 
images are then stored in a series of contiguous image 
ti lL Then the stored images are linked with a dyna.ic , 
self -executing program. The data structure can be 
15 created using a software program. Images are preferably 
first stored as bitmap files (.bmp). (Note that storing 
the resulting image files in the bitmap format is 
different from the bit mapping method of creating the 
ima ge files described herein.) An image 
20 program is used to convert the bitmap files to a JPEG 
{ jpg) format, which requires less storage space and 
consequently less time to display on a computer. The 

eJ creating the data structure can select how much 
detail to include in the conversion. JPEG images can be 
25 created for example, using 20 to 80% compression ratios 
of the original image. An advantage of the JPEG format 
Is that essentially empty tiles (tiles with mostly white 
or black space) compress down to very small files. 
Detailed files, however, do not compress as much. 
30 Additionally, the dynamic, self -executing program may 

include compression algorithms for displaying the entire 
image or portions thereof in the viewing window 

After downloading or installation of a data 
structure on a storage medium, when the user desires to 

• ™ he uses a mouse and "clicks" on the 
3 5 view the data images, he uses a 

icon for the self -executing data structure. The dynamic, 

self-executing program displays the image in a window. 
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Typically, the program will display a macro or thumbnail 
view of the entire specimen image at a lower 
magnification and a smaller window containing a 
particular image tile or groups of tiles at a higher 
5 magnification. The program enables the user to use the 
mouse or other pointing device to select a point or 
outline a region on the thumbnail view. The selected 
view will then be displayed in the smaller window at the 
second magnification. The user can move the mouse or 
10 pointing device and the image in the smaller window will 
scroll with the selection on the thumbnail view. In this 
way, the program simulates movement of a microscope slide 
under the field of view of the mechanical microscope. 
However, it should be noted that because of the 
15 one-to-one correspondence between the CCD pixels and the 

screen pixels, not all macro images may be. able to be 
,|- displayed on the monitor. The user may scroll through 

the macro image or select a compression feature to 
% display the entire macro image in the window. 
\° Another feature of the self -executing data 

structure is that when the image is displayed on the 
viewing screen, the user can select an image tile or sub- 
portion of the image and alternately view that portion of 
the image at each scanned magnification. For example, if 
25 the data was scanned at magnifications of 1.25X, 20X and 
40X, the user can "click" and see the same tile at each 
of those magnifications alternately. 

Brief De scription of the Drawings 

FIG. 1 is a block diagram of a system according 
30 to the invention for creating and transmitting locally, 
over an intranet or via the Internet data structures of 
an image of specimen on a microscope slide; 

FIG. 1A is representation of a microscope slide 
which has been arbitrarily assigned to be scanned into 
35 eighty tiled images; 
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F1G IB is a representation of the detected 
signals of the individual pixel sensors in a CCD optical 
array after detecting a selected image area to tile and 
the referenced data files containing the information 
5 describing the detected signals; 

FIG. 2 is a screen view of a system embodying 
the present invention showing a low magnification image 
of a specimen on a microscope slide in one window, a hxgh 
magnification image of a portion of the low magnification 
10 image selected by a region marker and a control window,- 
FIG . 3 is a view of a display screen of the 
apparatus embodying the present invention showing the 
control window a low magnification window having a 
plurality of high magnification micro image regions 
delineated therein and a high magnification window 
including one or more of the micro image regions ; 

FIG 4 is a view of a macro image of an actual 
breast cancer specimen displayed at 1.25X as seen on a 

computer monitor; 

FIG 5 is a view of the grid portion of FIG. 4 
outlining a region of interest selected by a pathologist 
displayed at 40X magnification; 

FIG 6 is a block diagram of the steps in the 
mapping of the scanned image from the optical sensor 
2S array to computer bit map in memory to the display on a 

user's monitor; 

FIG 7A is a file listing such as would be seen 

under Windows 95 file manager showing the data files 
included in a data structure for a breast cancer 
30 specimen; 

FIG. 7B is a file listing of a Java applet for 
controlling a data structure; 

FIG. 8 is file listing such as would be seen 
under Windows 95 file manager showing the data files 
included in an alternate data structure for a breast 
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FIGS. 9A and 9B are a block diagram of the 
apparatus embodying the present invention; 

FIG. 10 is a block diagram of a portion of the 
apparatus shown in FIG. 9 showing details of a mechanical 
arrangement of a microscope; 

FIG. 11 is a flow diagram related to operation 
of the apparatus; 

FIG. 12 is a flow diagram of details of one of 
the steps in FIG. 11; 

FIG. 13 is a display screen showing control 
parameters to be manipulated thereon; 

FIG. 14 is a flow chart for a region outlying 

routine ; 

FIG. 15 is a flow chart for a scanning and 
15 analyzing routine; 

FIG. 16 is a schematic showing of the limits of 
travel of the microscope stage with respect to the image 
tiles; 

FIG. 16A is a perspective view of the microscope 
20 stage and stepper motors and encoders providing a closed 
loop drive for the motors; 

FIG. 17 is a block diagram of a networked system 
allowing multiple workstations to obtain access to the 
microscope and to manipulate the microscope locally at 
25 each workstation; 

FIG. 17A is a view of the system described in 
connection with FIG. 10; and 

FIG. 18 is a block diagram of a remote networked 
system for distributing and accessing diagnostic images 
30 and data, i.e. virtual microscope slides, through a 

hypertext transport protocol based server directly or 
over a packet network. 

Detailed Description of the Preferred Embodiment 

FIG. 1 is a block diagram of a system according 
35 to the invention for creating, and transmitting over an 
intranet or via the Internet a virtual microscope slide, 
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i.e. interrelated data structures and display procedures 
depicting at multiple resolutions, images of a specimen 
on a microscope slide. The system includes a microscope 
with a digital platform for supporting the microscope 
slide. Digital platform or stage 11 has been specially 
calibrated to include a large number of increments for 
locating portions of specimen images with high precision. 
After calibration and initial registration of stage 11 in 
the microscope setup, a microscope slide or other 
substrate with a specimen to be scanned is placed on 
stage 11 . 

For exemplary purposes, the creation of virtual 
microscope slide specimen according to the invention will 
be described with respect to a breast cancer specimen. 
The first step in creating a data structure according to 
the invention is to establish a macro image of the entire 
specimen (or that portion of the specimen desired to be 
stored as the macro image) . The purpose for creating the 
macro or large area thumbnail image is to enable the 
viewer to see the entire specimen at once and to use the 
entire image to choose those significant portions thereon 
for viewing at greater magnification. In this example, 
the user has selected 1.25X as the magnification to 
display the entire breast cancer slide. Once specimen 
25 13a has been placed on stage 11, rotating optical 
assembly 15 are rotated to select lens 17 which 
corresponds to the 1.25X magnification. 

In accordance with the teachings of the prior 
patent application, the computer controlled microscope is 
moved to scan the entire image of specimen 13a. The 
focusing system is programmed to step through increments 
which detect/select only the high resolution center area 
of the field of view in order to avoid storing the fuzzy 
areas at the periphery of the field of view. In this 
example, the macro image will be stored in a 10 by 8 
array, for a total of 80 contiguous image tiles, as shown 
in FIG. 1A. 
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A typical microscope slide is about 77mm by 
25mm, where the usable area, without including the label, 
is about 57mm by 25m. Each of the 80 image segments is 
about 4 . 8mnv by 3.5mm in dimension. This means each of 
5 the 80 image segments will be scanned separately and 
stored as a separate image tile. 

The precision of the microscope systems is set 
up so that each step of the motor has a precision of . 1 
micron (micrometer) . In this example, the microscope is 
10 set up to move 48, 143 steps in the X direction and 35,800 
steps in the Y direction at 1.25X magnification for each 
of the 80 image areas. At higher magnifications, the 
image areas to scan are considerably smaller, so the 
number of steps is corresponding smaller. For each of 
15 the 80 image areas, the microscope lens will detect only 
the high resolution center area of the field of view. 

The optical image of the desired image area is 
then detected by optical array sensor 19 (preferably a 
CCD sensor array) . In this example, each of the 8 0 
20 scanned areas is sensed by the entire array, which 

includes 752 pixels by 480 pixels. The optical array 
sensor sends electrical signals indicative of the 
detected image to microscope controlled computer 32. 
Computer 3 2 stores the scanned images, including the top 
25 left X-Y stage coordinates for each of the 80 individual 
areas of the microscope slide. Each of the 80 scanned' 
image areas' pixel locations are stored in a bit mapped 
file (i.e., a file which contains a map of the location 
of each bit in the area) which corresponds to the layout 
3 0 of the individual images thereon. Thus, all of the 

pixels from the image tile derived from region A on FIG. 
1A, which is the seventh from the left and in the top 
row, are individually assigned unique locations in the 
computer memory's bit-mapped file (FIG. 6), and are also 
35 stored in the data structure image tile file as shown in 
FIG. IB. 
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Each of the stored data image tiles is a 
standard image file with extension.bmp, and is of the 
order of one megabyte, i.e. each of the 752 x 480 pixels 
is stored as 3 bytes of red, green and blue image data 
(752 x 480 x 32 = 1,082,880 bytes). Since the location 
of each image tile is known according to the bitmap, the 
complete microscope image can be recreated by painting 
(displaying) each image tile in accordance with its grid 
location. It should be noted 

To display the resulting image, computer 32 
calculates the appropriate portion to be displayed from 
each image tile depending upon the relative size of the 
display screen. Since the stored image data is usually 
greater than the size of the typical monitor, the viewer 
must scroll through the image on the window to view it 
entirely- However, an optional compression algorithm can 
be used to compress the entire image into the viewing 
window. The X-Y coordinate information is used by the 
viewing and manipulation program to reconstruct the image 
tiles into a complete image of the specimen. The 
resulting image is larger, and with better resolution 
than would be achieved if optics technology were able to ^ 
construct a single lens capable of viewing the entire 
specimen in one field of view. In this example, each of 
the 80 image tiles has digital resolution of 752 x 480 
pixels, with corresponding optical resolution of 
approximate .2 microns at 40X to approximately 6.4 

microns at 1.25X. 

After the macro or thumbnail images are 
digitally scanned and stored with their X-Y coordinate 
information, the user then examines the macro image or 
original specimen for significant details. Typically, 
the user will highlight with a marking pen the areas to 
be viewed at higher magnification. The user then changes 
35 the magnification of optics system 15 to the desired 

higher magnification, moves the scanning system to bring 
the selected region into view. Computer 32 then repeats 
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the scanning and image tile creation process for the 
selected region, but at higher magnification and with a 
new grid system to locate the scanned selected regions. 

In the example, the user has selected region B 
5 shown on FIG. 1A to perform a second view at a higher 
magnification. The user selects, for example, a 40X 
magnification. The computer calculates the number of 
tiles to cover the selected area at 40X magnification and 
sets up a second grid. 
10 J t should be noted that region B crosses over 

several of the larger tiles in FIG. 1A. Because of the 
extreme precision of the instrument, 0.1 micron 
resolution, locating such selected regions with high 
resolution is readily accomplished. As noted above, the 
15 compute calculates the size of the image portion, in this 
case as an example, X = 1500 and Y = 1200 stepping 
increments. Each image portion at the 40X resolution is 
detected by the optical sensor array, 752 by 480 pixels. 
Each resulting data file is stored in a separate, high 
20 magnification mapped area of memory so that the computer 
can easily recall the location of region B, or any of its 
200 individual image tiles, when requested by a user. 

Once the user has completed selecting and having 
the computer controlled microscope system scan and store 
25 the digital images in image tiles, computer 32 stores the 
mapped . btnp files along with their coordinate information 
and creates slide image data structure 31 in FIG. l. 
Slide image data structure includes all of the bitmap 
image tile files at both magnifications (note that 
30 similarly, additional images could be stored at further 
magnifications, if desired), as well as X-Y coordinate 
information for the location of the various image tiles. 

FIG. 7A is a file listing such as would be seen 
under a Windows 95 file manager showing the data files 
35 included in a data structure for a breast cancer 

specimen. Included in the file listing are FinalScan.ini 
and SlideScan.ini as well as sixty bitmap data files. 
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Slidescan.ini is a listing of all the original bitmap 
(.bmp) files. The bitmap files represent the individual 
image tiles in the scan at, say, 1.25X magnifications. 
Slidescan.ini is set forth below in Table 1 and describes 
the X-Y coordinates for each image tile file. When the 
data structure is viewed by a control program, the 
program uses the X-Y coordinates to display all the image 
tiles contiguously . 
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[Header] 
x=278000 
y=142500 
5 lXStepSize=4 814 3 
lYStepSize=3 58 00 
iScannedCount=3 7 
[Ssl] 
X=181714 
10 y=142500 
ISs2) 
x=133571 
y=142500 
[Ss3] 
15 x=37285 
y=106700 
tSs4) 
x=85428 
y=1067O0 
20 [Ss5] 

x=133571 
y=106700 
[Ss6] 
X=181714 
: 25 y=106700 
[Ss7] 
x=229857 
y=106700 
[Ss8] 
30 x=r229857 
y=70900 
[Ss9] 
x=181714 
y=70900 
35 [SslO] 

x=133571 
y=70900 
[Ssll] 
x=85428 
40 y=70900 
[Ssl2] 
x=37285 
y=70900 
[Ssl3] 
45 x=-10858 
y=70900 
[Ssl4] 
x=- 10858 
y=35100 
50 (Ssl5) 
x=37285 
y=35100 
[Ssl6] 
x=85428 
55 y=35100 



TABLE 1 Slidescan.ini 

[Ssl7] 
X=133571 
y=35100 
[Ssl8] 
x=181714 
y=35100 
[Ssl9] 
x=229857 
y=35100 
[Ss20] 
x=278000 
y=-700 
[Ss21] 
x=229857 
y=-700 
[Ss22] 
x=181714 
y=-700 
[Ss23] 
X=133571 
y=-700 
(Ss24) 
X=85428 
y=-700 
[SS25] 
X=37285 
y=-700 
[Ss26] 
x=-10858 
y=-700 
[Ss27] 
x=-10858 
y=-36500 
[Ss28] 
x=37285 
y=-36500 
fSs29] 
x=85428 
y=-36500 
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[Ss30] 
x= 133571 
y = - 36500 

[Ss3l] 
5 x=181714 

y=-36500 

[Ss32] 

x=229857 

y=-36500 
10 [Ss33] 

x=278000 

y= -36500 

[Ss34] 

x=278000 
15 y=-72300 

[SS35] 

x=229857 

y=-72300 

[SS36] 
20 X=1B1714 

y=-72300 

[SS37] 

x=133571 

y=-72300 

25 Table 2 is a listing of the file FinalScan.ini, which is 
a listing the X-Y coordinates of the high magnification 
image tiles scanned and stored. 



TABLE 2 - FinalScan.ini 

[Header] 
3 0 tPat ient ID=mda02 7 
tAccession= 
tOperatorID= jwb 
tTimeOf Scan=8/4/97 1:19:56 

PM 

3 5 lXStageRef =2 78 00 0 

lYStageRef =14 2 50 0 
iImageWidth=75 2 
iImageHeight=4 8 0 
lXStepSize=1590 

4 0 lYStepSize=1190 

lXOf f set=-1900 
lYOf f set=-400 
dMagnif ication=4 0 
lAnalysisImageCount=10 5 
4 5 lCalibrationImageCount=0 

[DaO] 
x=214532 
y=655B4 

[Dal] 
50 x=2!2996 



y=65584 
[Da23 
x=211460 
y=65584 
[Da3] 
x=209924 
y=65584 
[Da4] 
x=208388 
y=65584 
[Da5] 
x=206852 
y=65584 
[Da6] 
x=205316 
y=65584 
[Da7] 
x=203780 
y=65584 
[Da8] 
x=214532 
y=64400 
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[Da9] 

x=212996 

y=64400 
[DalO] 

x^211460 

y=64400 
[Dall] 

x=209924 

y=64400 

[Dal2] 

x=208388 

y=64400 

[Dal3] 

x=206852 

y=6440O 

[Dal4 J 

x=205316 

y=64400 

[Dal5J 

x=203780 

y=64400 

[Dal6] 

x=214532 

y=63216 
: lDal7] 
x=212996 
y=63216 

[Dal8] 
x=211460 
y=63216 

[Dal9] 
x=209924 
y=6321€ 

[Da20] 
x=208388 
y=63216 

[Da21) 
x=206852 
y=63216 

[Da22] 
x=205316 
y=63216 

[Da23] 
x=203780 
y=63216 

[Da24] 
x=214532 
y=62032 

[Da25] 
x=212996 
y=62032 

[Da26) 
x=211460 
y=62032 

[Da27] 



x=209924 

y=62032 
[Da28j 

x=208388 

y=62032 
[Da29] 

x=206852 

y«62032 
[Da30] 

x=205316 

y=62032 
[Da31] 

x=203780 

y=62032 
[Da32] 

x=214532 

y=60848 

[Da33] 

x=212996 

y=60848 

[Da34] 

x=211460 

y=60848 

[Da35] 

x=209924 

y=60848 

[Da3 6] 

x«208388 
y=60848 

[Da37] 
x=206852 
y=60848 

[Da38] 
x=205316 
y=60848 

[Da39] 
x=203780 
y=60848 

[Da40) 
x=214532 
y=59664 

[Da41] 
x=212996 
y=59664 

{Da42] 
x=211460 
y=59664 

[Da4 3] 
x=209924 
y=59664 

[Da44] 
x=208388 
y=59664 

[Da45] 
x=206852 
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y=59664 
[Da46] 
X=205316 
y=59664 
5 [Da47] 

x=203780 
y=59664 
[Da48] 
x=214532 
10 y=58480 
[Da49] 
x=212996 
y=58480 
[Da50] 
15 x=211460 
y=58480 
[DaSl] 
x=209924 
y = 584 80 
20 [Da52] 

x=208388 
y=58480 
[Da53] 
x=206852 
25 y=58480 
[Da54] 
X =205316 
y=58480 
[Da55] 
30 x=2O3780 
y^584 80 
(Da56) 
x=180740 
y=82160 
35 [Da57] 

x=l79204 
y=82160 
[Da58] 
X=177668 
40 y=82l60 
[Da59] 
X =176132 
y=B2160 
[Da60] 
45 x=174596 
y=82160 
[Da61] 
X=173060 
y=82l60 
50 [Da62] 

X=171524 
y=82l60 
[Da63] 
X =180740 
55 y=80976 
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[Da64] 
X =179204 
y=80976 
[Da65) 
X=177668 
y=80976 
[Da66] 
X=176132 
y=80976 
[Da67] 
X=174596 
y=80976 
[Da68] 
x=173060 
y=80976 
[Da69] 
x=171524 
y=80976 
[Da70] 
X=180740 
y=79792 
[Da71] 
x=179204 
y=79792 
[Da72] 
X=177668 
y=79792 
[Da73] 
x-176132 
y=79792 
[Da74] 
x=174596 
y=79792 
[Da75] 
x=173060 
y=79792 
[Da76] 
x=171524 
y=79792 
[Da77] 
X=180740 
y=78608 
[Da78] 
X=179204 
y=78608 
(Da79] 
X=177668 
y=78608 
[Da80] 
X=176132 
y=78608 
[Da81] 
x=l74596 
y=78608 
[Da82] 
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x=173 060 
y=78608 
[Da83] 



y=75056 

[DalOl] 
x=176132 
y=75 05 6 

[Dal02] 
x=174596 
y=75056 

[Dal03] 
x=173060 
y=75056 

[Dal04] 
x=171524 
y=75056 



x=171524 
5 y=78608 
(Da84] 



x=180740 
y=77424 
[Da85] 



10 



X=179204 
y=77424 
[Da86] 



x=17766 8 



y=77424 
15 [Da87] 

x=176132 
y=77424 
[Da88] 
X*174596 
20 y=77424 
[Da89] 
x=173060 
y=77424 
[Da90] 
25 i; x=171524 
y=77424 
[Da91] 
\ X=180740 
.\ y=76240 
30 [Da92] 

x=179204 
y=76240 
[Da93] 
x=177668 
35 y=76240 
[Da94] 
x=176132 
y=76240 
[Da95] 
40 x=174596 
y=76240 
[Da96] 
x=173060 
y=76240 
45 [Da97] 

x=171524 
y=76240 
[Da98] 
x=180740 
50 y=75056 
[Da99] 
x=179204 
y=75056 
[DalOO] 
55 x=177668 
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Computer 32 can also use the scanned image files 
to create a self -executing data structure. By 
compressing the . bmp images to .jpg and adding a dynamic, 
self-executing program which enables the user to view, 
reconstruct and manipulate the image tiles, the user can 
use the data structure as a virtual microscope slide of 
the original specimen. Preferably, the dynamic, self- 
executing program is a Java applet, such as shown on FIG. 

E computer 32 can provide the slide image data 

structure 31 directly or via an intranet browser 33 to 
local viewer 34, or via an Internet server 38. Slide 
image data structure 37 is shown as being directly 
accessible from Internet server 38. Alternatively, a 
user can download the slide image data structure on his 
own computer 39, use an internet browser 43 and view the 
reconstructed images. Another alternative is for 
computer 32 to store the slide image data structure on a 
CD-rom, Jazz drive or other storage medium. 

To view slide image data structure 31 or 37, the 
use r who for example, has acquired the data structure 
via a CD-rom, first installs the CD-rom in the CD-rom 
drive of his computer. Then the user opens up a browser 
or other applications program which can read the Java 
applet installed on the CD-rom with the image tiles. 
Note that in some instances no separate browser program 
may be required. In some case, the CD-rom may include 
the complete applications program for viewing, 
reconstructing and manipulating the image tiles. In the 
instant example, the user will then select the icon or 
file listing for the slide image data structure and the 
control program will display the data files. 

FIG. 2 is a screen view of a system embodying 
the present invention showing a low magnification image 
24 of a specimen on a microscope slide in one window, a 
high magnification image 26 of a portion of the low 
magnification image selected by a region marker 30 and a 
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control window 28. FIG. 3 is a view of a display screen 
of the apparatus embodying the present invention showing 
the control window 28, a low magnification window 24 
having a plurality of high magnification micro image 
5 regions 310 delineated therein and a high magnification 
window 26 including one or more of the micro image 
regions 310, 314, 316. FIG. 4 is a view of a macro image 
of an actual breast cancer specimen displayed at 1.25X as 
seen on a computer monitor. FIG . 5 is a view of the grid 
10 portion of FIG. 4 outlining a region of interest selected 
by a pathologist displayed at 40X magnification. 

Recall that region A in FIG. 1A was about 4.8mm 
by 3.5mm. This area creates 752 by 480 pixels of sensed 
data, or 360, 930 pixels of information. Each pixel sends 
15 information about its location and the image it sensed to 
the computer. The computer stores this information in a 
jseries of data files (typically .bmp format, but .tif or 
\gif could also be used) . Thus, it can be seen that 
^several more pixels of sensed data are available for 
20 "viewing on a computer monitor operating at 640 by 480. 
To view the entire image, the user must scroll through 
the image tiles. However, scrolling need not be done on 
a tile, by tile basis. Rather, the user scrolls by 
pointing to a pixel on the monitor. 
2 5 Figure 6 is a block diagram showing how the 

control program locates and scrolls through the stored 
image tiles. Using the example from Figure la, a 
complete data structure has been created. When the user 
loads the data structure (of the microscope slide) into 
30 his personal computer or views it from an Internet 

browser, the control program recreates a bit map of the 
stored data. The bit map of the entire slide is shown in 
Figure 6. Image tile A is also high-lighted. This bit 
map enables a user to point to or otherwise reference a 
35 location on the slide. 

The X-Y coordinate information specified in the 
data structure enables X-Y translation of the specific 
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image tiles and specific pixels within the image tile. 
When the control program first loads the image, because 
this image file is so large, only a small number of the 
available tiles are displayed in the active window on the 
user's monitor. The user uses his mouse or pointing 
device to scroll through the active window to view the 
entire macro image. The X-Y coordinate information 
selected by the mouse translates into specific image 
tiles or portions therein. The computer takes the mouse 
pointer information and retrieves the image data from the 
series of stored tile images and displays them on the 
monitor for viewing the by user. 

Because of the large amount of CCD pixel 
information stored, actual CCD pixel information can be 
15 recreated in the viewing window. The entire system 
operates in a loop, where the user inputs a mouse 
location, the computer translates the mouse location from 
the screen coordinates (screen pixels) to the X-Y 
coordinates on the bit map. 

Similarly, the user may select the high 
magnification data images. These are outlined by a dark 
grid, indicating the areas stored. The user operates the 
mouse in the same manner as described above. The control 
program locates the stored X-Y coordinates and retrieves 
the selected parts of the image, CCD stored pixel by CCD 

stored pixel . 

As mentioned above, to save storage space, 
computer 32 can perform a data compression on each of the 
image tile files. A preferred data compression is JPEG , 
which is readily transferred and recognized by most 
Internet browser programs. Also, JPEG allows flexibility 
in the amount of data to be compressed, from 20 to 80 
percent. FIG. 8 is file listing such as would be seen 
under Windows 95 file manager showing the data files 
35 included in an alternate data structure, one in which the 
data files have been compressed or converted to JPEG 
(jpg) format for a breast cancer specimen. The file 



20 



25 



30 



WO 98/39728 



PCT/US98/04011 



-33- 

index.html (shown in Table 3) is the listing which 
contains the X-Y coordinate information for these data 
files. This is the information that is read by the 
dynamic, self -executing program for viewing, 
reconstructing and manipulating the image tiles into the 
macro and micro views . 
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TABLE 3 -- index.html 

<HTML> 
<TITLE> 

DCIS_027 - Web Slide 
</TITLE> 

< APPLET CODE=WebSlide/BliWebSlide. class NAME=DCIS 02 7 
WIDTH-3384 HEIGHT=960 HSPACE = 0 VSPACE=0 AJ_jIGN = Middle > 
"tPatientID" VALUE = "mda027"> 
"tAccession" VALUE = " " > 
"tOperatorlD" VALUE = "jwb"> 

"tTimeOfScan" VALUE = "8/4/97 1:19:56 PM"> 
"lXStageRef" VALUE = " 27 8 00 0" > 
"lYStageRef" VALUE = " 142500" > 
"ilmageWidth" VALUE = "752" > 
"ilmageHeight" VALUE = "480"> 
"lXStepSize" VALUE = "1590"> 
"lYStepSize" VALUE = "1190"> 
"lXOffset" VALUE = "-1900"> 
"lYOffset" VALUE = "-400"> 
"dMagnif icatrion" VALUE = "4 0" > 
"ilmageCount" VALUE = "105"> 
» lXSsStepSize" VALUE = "4 8143"> 
"lYSsStepSize" VALUE = "3 5800"> 
"iScannedCount" VALUE - "37"> 
"lStartX" VALUE 
"IStartY" VALUE 
"Ssl X" VALUE = 



< PARAM NAME * 
10 < PARAM NAME * 

< PARAM NAME : 
< PARAM NAME : 
< PARAM NAME ' 
< PARAM NAME : 
15 < PARAM NAME ■■ 

< PARAM NAME •■ 

< PARAM NAME 
< PARAM NAME 
< PARAM NAME 

2 0 < PARAM NAME 
< PARAM NAME 
< PARAM NAME 
< PARAM NAME 

< PARAM NAME 

2 5 < PARAM NAME 

< PARAM NAME 

< PARAM NAME 
< PARAM NAME 
< PARAM NAME 

3 0 < PARAM NAME 

< PARAM NAME 

< PARAM NAME 

< PARAM NAME 
< PARAM NAME 

3 5 < PARAM NAME 

< PARAM NAME 
< PARAM NAME 
< PARAM NAME 
< PARAM NAME 

4 0 < PARAM NAME 

< PARAM NAME 
< PARAM NAME 
< PARAM NAME 
< PARAM NAME 

4 5 < PARAM NAME 

< PARAM NAME 

< PARAM NAME 
c PARAM NAME 
< PARAM NAME 

5 0 < PARAM NAME 

< PARAM NAME 
< PARAM NAME 
< PARAM NAME 

< PARAM NAME 
55 < PARAM NAME 



_ H 



Ssl 
Ss2~ 
Ss2~ 
Ss3' 
Ss3 
Ss4* 
Ss4* 
Ss5" 
Ss5* 
■Ss6 
■Ss6" 
•Ss7 
'Ss7 
Ss8 
Ss8 
Ss9 
Ss9 



Y" 
X " 
"Y " 
"X" 
Y" 
~X" 

y " 

X " 
Y" 

I x " 
Y" 
X" 

~Y " 

X 

_X" 
Y" 



"SslO 
"SslO" 
"Ssll 
"Ssll 
"Ssl2 
"Ssl2 
"Ssl3 
"Ssl3 
"Ssl4 
"Ssl4 



X" 

~y if 
~X M 

x M 
y« 

X" 
~Y" 

~x " 

Y" 



VALUE 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE = 
VALUE 
VALUE 
VALUE 
VALUE 
VALUE 
VALUE 
VALUE 
VALUE 
VALUE 
VALUE 



= "278000 " > 
= "142500"> 
"181714"> 
"142500"> 
"133571" > 
"142500" > 
" 3 7 2 8 5 " > 
"106700" > 
"85428" > 
"106700"> 
"133571"> 
"106700"> 
"181714 " > 
" 106700 "> 
"229857"> 
"106700"> 
"229857" > 
"70900" > 
" 1 8 1 7 1 4 " > 
"70900" > 
= "133571" > 
= "70900"> 
= "854 2 8" > 
« "70900"> 
= " 3 7285" > 
= "70900"> 
= "-10858"> 
m ,, 70900"> 
= »-10858 M > 
= "35100"> 
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< PARAM NAME = M Ssl5_X" VALUE = "37285"> 
< PARAM NAME = "Ssl5__Y M VALUE = ••35100"> 

< PARAM NAME = "Ssl6_X" VALUE = " 85428 "> 
< PARAM NAME = "Ssl6_Y" VALUE = "35100 n > 

5 < PARAM NAME = "Ssl7_X M VALUE = 11 133571" > 
< PARAM NAME = "Ssl7_Y" VALUE = "35100"> 

< PARAM NAME = "Ssl8_X ,! VALUE = ,, 181714 ,, > 
< PARAM NAME = "Ssl8__Y" VALUE = "3 5100"> 

< PARAM NAME = "Ssl9_X" VALUE = "229857"> 
10 < PARAM NAME = "Ssl9_Y n VALUE = "35100"> 
< PARAM NAME = "Ss20_X" VALUE = "278000"> 

< PARAM NAME = "Ss2 0_Y" VALUE = " -7 00" > 

< PARAM NAME = "Ss21_X" VALUE = "229857" > 
< PARAM NAME = "Ss21_Y" VALUE = "~700"> 

15 < PARAM NAME = "Ss22_X" VALUE = " 181714 "> 
< PARAM NAME = "Ss22_Y" VALUE = "-700"> 
< PARAM NAME = "Ss23_X" VALUE = " 133571 "> 
< PARAM NAME = ,, Ss23_Y" VALUE = M -700"> 
< PARAM NAME = "Ss24_X M VALUE = " 85428 "> 

20 < PARAM NAME = "Ss24_Y" VALUE = "-700"> 
< PARAM NAME = "Ss25_X" VALUE = " 37285 "> 
< PARAM NAME = "Ss25_Y" VALUE = "-700"> 
< PARAM NAME = "Ss26_X" VALUE = " -10858 "> 
< PARAM NAME = M Ss26_Y" VALUE - H -700"> 

25 . . < PARAM NAME « "Ss27_X" VALUE = "-10858"> 
< PARAM NAME = "Ss27_Y" VALUE = " -36500 "> 
< PARAM NAME = "Ss28_X" VALUE = "37285" > 
< PARAM NAME = "Ss28__Y" VALUE = "-36500"> 
< PARAM NAME = "Ss2 9_X" VALUE = "854 2 8"> 

3 0 < PARAM NAME = "Ss29_Y" VALUE = » -36500" > 

< PARAM NAME = "Ss3 0_X" VALUE = « 13 3 5 71 "> 
< PARAM NAME = ,, Ss30_Y" VALUE = "-36500" > 
< PARAM NAME = "Ss31_X M VALUE = "181714"> 
< PARAM NAME = "Ss31__Y" VALUE «= "-36500"> 
35 < PARAM NAME = "Ss32_X" VALUE = "229857"> 
< PARAM NAME = "Ss32_Y" VALUE = " -36500 "> 
< PARAM NAME = "Ss33__X" VALUE = "278000" > 
< PARAM NAME = "Ss33_Y" VALUE = " -36500" > 
< PARAM NAME = "Ss34_X" VALUE « " 278000 "> 

4 0 < PARAM NAME = "Ss34_Y" VALUE = "-723 00"> 

< PARAM NAME = "Ss3 5_X" VALUE = "229857"> 
< PARAM NAME = "Ss35_Y" VALUE = " -72300" > 
< PARAM NAME = "Ss36 X" VALUE = "181714 "> 





< PARAM 


NAME 




"Ss36 Y 


" VALUE 


= "-72300"> 


45 


< PARAM 


NAME 




"Ss37 X 


" VALUE 


= "133571"> 




< PARAM 


NAME 




"Ss37 Y 


" VALUE 


= "-72300"> 




< PARAM 


NAME 




"DaO X" 


VALUE = 


"214532" > 




< PARAM 


NAME 




"DaO Y" 


VALUE = 


"65584 "> 




< PARAM 


NAME 




"Dal X" 


VALUE = 


"212996" > 


50 


< PARAM 


NAME 




"Dal Y" 


VALUE = 


"65584 "> 




< PARAM 


NAME 




"Da2 X" 


VALUE = 


"211460"> 




< PARAM 


NAME 




"Da2 Y" 


VALUE = 


"65584 »> 




< PARAM 


NAME 




"Da3 X" 


VALUE = 


"209924"> 




< PARAM 


NAME 




"Da3 Y" 


VALUE = 


"65584 "> 


55 


< PARAM 


NAME 




"Da4 X" 


VALUE = 


"208388"> 
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< PARAM 

< PARAM 

< PARAM 

< PARAM 
5 < PARAM 

< PARAM 

< PARAM 

< PARAM 

< PARAM 
10 < PARAM 

< PARAM 

< PARAM 

< PARAM 
< PARAM 

15 < PARAM 

< PARAM 

< PARAM 

< PARAM 

< PARAM 
2 0 < PARAM 

< PARAM 

< PARAM 

< PARAM 
< PARAM 

2 5 < PARAM 

< PARAM 

< PARAM 

< PARAM 

< PARAM 

3 0 < PARAM 

< PARAM 

< PARAM 

< PARAM 
< PARAM 

3 5 < PARAM 

< PARAM 

< PARAM 

< PARAM 

< PARAM 

4 0 < PARAM 

< PARAM 

< PARAM 

< PARAM 

< PARAM 

4 5 < PARAM 

< PARAM 
< PARAM 

< PARAM 

< PARAM 
50 < PARAM 

< PARAM 
< PARAM 
< PARAM 

< PARAM 

5 5 < PARAM 



NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME = 
NAME « 
NAME * 
NAME = 
NAME 
NAME = 
NAME . 
NAME » 
NAME j 
NAME « 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 
NAME 



" Da4_Y " VALUE = 
"Da5_X" VALUE = 
"Da5_Y" VALUE = 
'*Da6_X" VALUE = 

" Da 6 Y " VALUE = 

"Da7_X" VALUE = 

" Da 7 Y " VALUE = 

n Da8_X" VALUE = 
" Da B_Y " VALUE = 
"Da9_X" VALUE = 

" Da 9 Y " VALUE = 

»DalO__X'* VALUE = 
"DalO_Y" VALUE = 
"DallJ" VALUE = 

" Da 1 1 Y " VALUE = 

"Dal2__X" VALUE = 

" Da 1 2 Y " VALUE = 

••DalB^X" VALUE = 

" Da 1 3 Y M VALUE = 

"Dal4_X" VALUE = 

11 Dal 4 Y " VALUE = 

"Dal5_X" VALUE = 
,, Dal5_Y" VALUE = 
"Dal6_X" VALUE = 

" Dal 6 Y M VALUE 

"Dal7_X" VALUE 
"Dal 7_Y " VALUE 
«Dal8_X" VALUE 
" Dal8_Y 
"Dal9_X 

" D a 1 9 _Y " VALUE 
"Da2 0_X" VALUE 
"Da20_Y" VALUE 
"Da21_X" VALUE 
"Da21_Y" VALUE 
»Da2 2_X" VALUE 

" Da 2 2 Y M VALUE 

"Da2 3_X" VALUE 
"Da23_Y" VALUE 
"Da24_X" VALUE 

" Da 2 4 Y " VALUE 

"Da2 5_X" VALUE 
"Da2 5_Y" VALUE 
«Da2 6_X" VALUE 

•• " Da 2 6 Y M VALUE 

• "Da2 7_X" VALUE 
: "Da2 7_Y" VALUE 
: "Da2 8_X" VALUE 

: " Da 2 8 Y H VALUE 

: "Da2 9_X M VALUE 
: "Da2 9_Y" VALUE 
= n Da3 0_X" VALUE 

= " Da 3 0 Y M VALUE 

= "Da31_X" VALUE 
= M Da31 Y" VALUE 



VALUE 
VALUE 



"65584 " > 
"206852 " > 
"65584 " > 
"205316 " > 
"65584 " > 
"203780" > 
"65584 " > 
"214532" > 
" 6 4 4 0 0 " > 
"212996 " > 
"64400" > 
"211460 n > 
"64400 ,, > 
"209924 " > 
"644 00 M > 
" 2 0 8 3 8 8 " > 
"644 00"> 
" 2 06 8 52 "> 
= « 644 00" > 
= "205316"> 
= M 6 4 4 0 0 " > 
= "203780 n > 
= " 64400" > 
= "214532"> 
= " 6 3216 M > 
= " 212996 "> 
= " 6 3 2 1 6 " > 
= "2114 60 "> 
= "63216"> 
= "209924" > 
= " 6 3216 " > 
= "208388" > 
= "63216"> 
= "206852"> 
= " 6 3 2 1 6 " > 
= "205316"> 
= "63216"> 
= "203780"> 
= "63216"> 
= "214532"> 
= "62032"> 
= "212996 "> 
= "62032" > 
= "211460"> 
= "62032"> 
= "209924"> 
= "62032"> 
= "208388 " > 
= "62032"> 
= »206852"> 
= "6 203 2" > 
= "205316"> 
= "6 2032" > 
= "2 037 80" > 
= "62032"> 
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<PARAM 


NAME 




"Da3 2 


X" 


VALUE 




"214532"> 




< PARAM 


NAME 


= 


11 Dei 3 2 


Y" 


VALUE 


= 


"60848" > 




<PARAM 


NAME 


= 


"Da3 3 


X" 


VALUE 


= 


"212996" > 




< PARAM 


NAME 


= 


"Da3 3 


Y" 


VALUE 




"60848"> 


5 


< PARAM 


NAME 


= 


"Da3 4 


X" 


VALUE 




"211460" > 




< PARAM 


NAME 




"Da3 4 


Y " 


VALUE 


= 


"6 084 8" > 




< PARAM 


NAME 


= 


"Da3 5* 


X" 


VALUE 




"209924"> 




< PARAM 


NAME 


= 


"Da 3 5' 


Y" 


VALUE 




"6084 8" > 




< PARAM 


NAME 


= 


" Da3 6' 


X" 


VALUE 


.a. 


"2083 88" > 


10 


< PARAM 


NAME 


= 


" Da3 6* 


Y " 


VALUE 


= 


"60848" > 




< PARAM 


NAME 


= 


"Da 3 1 


X" 


VALUE 


= 


"206852"> 




< PARAM 


NAME 




"Da3 7* 


"Y" 


VALUE 


= 


"60848"> 




< PARAM 


NAME 


= 


"Da38 


X" 


VALUE 


= 


"205316"> 




< PARAM 


NAME 


as 


"Da3 8* 


Y M 


VALUE 




M 60848"> 


15 


< PARAM 


NAME 


= 


"Da 3 9" 


X" 


VALUE 


= 


"203780"> 




< PARAM 


NAME 




"Da 3 9" 


yti 


VALUE 




"6 084 8" > 




< PARAM 


NAME 


= 


"Da4 0* 


"X " 


VALUE 


= 


"214532 " > 




< PARAM 


NAME 


= 


"Da4 0* 


"Y" 


VALUE 




"59664"> 




< PARAM 


NAME 




"Da4l" 


X" 


VALUE 




"212996"> 


20 


< PARAM 


NAME 


= 


"Da4l" 


~Y " 


VALUE 




"59664"> 




< PARAM 


NAME 




"Da4 2" 


~X »• 


VALUE 


= 


"211460"> 




< PARAM 


NAME 


= 


"Da4 2* 


Y" 


VALUE 




"59664"> 




< PARAM 


NAME 


— 


" Da4 3" 


~x " 


VALUE 




"209924 "> 




< PARAM 


NAME 


= 


"Da4 3" 


"Y" 


VALUE 


= 


"5 96 64 "> 


25 


• < PARAM 


NAME 


= 


"Da44~ 


X" 


VALUE 


s= 


"208388" > 




< PARAM 


NAME 




" Da4 4~ 


"yi. 


VALUE 


ss 


"5 96 64 "> 




< PARAM 


NAME 




"Da4 5* 


~X" 


VALUE 


S 


"206852"> 




< PARAM 


NAME 




"Da4 5~ 


"y w 


VALUE 


= 


"59664 "> 




< PARAM 


NAME 




" Da4 6~ 


~x » 


VALUE 


= 


"205316"> 


30 


< PARAM 


NAME 




"Da4 6~ 


"Y" 


VALUE 


= 


"59664 "> 




< PARAM 


NAME 


= 


"Da4 7~ 


*X " 


VALUE 




" 2 03 780 "> 




< PARAM 


NAME 




" Da 4 7" 


~Y" 


VALUE 


cr 


"59664"> 




< PARAM 


NAME 




"Da4 8" 


~X » 


VALUE 


= 


"214532"> 




< PARAM 


NAME 


= 


"Da4 8~ 


Y" 


VALUE 


= 


"58480"> 


35 


< PARAM 


NAME 


= 


"Da4 9~ 


"X" 


VALUE 




"212996"> 




< PARAM 


NAME 




"Da4 9~ 


Y" 


VALUE 




"58480"> 




< PARAM 


NAME 




"Da50~ 


"X" 


VALUE 


= 


"211460"> 




< PARAM 


NAME 




"Da50" 


~Y" 


VALUE 




"58480"> 




< PARAM 


NAME 




"Da51~ 


"x " 


VALUE 


= 


"209924"> 


40 


< PARAM 


NAME 




"Da51~ 




VALUE 


= 


" 58480 " > 




< PARAM 


NAME 




"Da52_ 


X " 


VALUE 




"208388" > 




< PARAM 


NAME 




" Da52_ 


y ii 


VALUE 




" 5 8 4 8 0 " > 




< PARAM 


NAME 




Da53_ 


X " 


VALUE 




ii o n r o r i ik 

"206852 > 




^ HTi D 7S Kit 


NAMc* 




Da 5 3 


y n 


VALUE 




It C Q Jt O A II ^ 

5 8480 > 


4 D 


< ir AX-ftJYl 


"KT AMP 




uab4 


~V It 


t T T TP 

VALUE 








< PARAM 


NAME 




"Da54" 


Y" 


VALUE 




"58480"> 




< PARAM 


NAME 




"Da 5 5"" 


"X" 


VALUE 




"203780"> 




< PARAM 


NAME 




"Da55~ 


~Y" 


VALUE 




"58480" > 




< PARAM 


NAME 




"Da56~ 


X" 


VALUE 




"180740" > 


50 


< PARAM 


NAME 




"Da 5 6' 


Y" 


VALUE 




"82160"> 




< PARAM 


NAME 




" Da57 


"X" 


VALUE 




"179204"> 




< PARAM 


NAME 




" Da57~ 


■y,i 


VALUE 




"82160"> 




< PARAM 


NAME 




"Da5 8 


X" 


VALUE 




"177668"> 




< PARAM 


NAME 




"DaSS"*" 


Y " 


VALUE 




"82160" > 


55 


< PARAM 


NAME 




"Da59 


X" 


VALUE 




"176132"> 



WO 98/39728 



PCT7US98/0401 J 



38- 



10 



IS 



20 



25 



30 



35 



40 



45 



50 



55 



< PARAM 

< PAR-AM 
< PARAM 

< PARAM 

< PARAM 

< PARAM 
< PARAM 
< PARAM 
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- Ho drawings, and especially to 
Keferring now to the d * 1o „ 

„»s. 9». ana !0. apparatus '«^ -ieroKOpio im , g es 
unification ana high mag" £««a» 

is S « r r ; , ; ^^""cludes a 12 which is • 

5 numeral 10. Tne sy in combination with a 

aual Pentium Pro persona! compute _ 
Hitachi HV-C20 video camera 14 ^ t . abl . 

^ioplan 2 microscope ™ „ which captures 

„ receive signals from the camera , slid , „ 

10 light from the microscope 16 havrng ^ 

, TTinT encoded motorized ^<-ay^ 

p osition.d on an » _ ^ 2000 s „ ge 

encoded motonred stage 20 r « po „se to the 

controller for contrcX-nc the^ , ^ ^ 

computer 22 * » ^ fay ^ „ ior os=ope and 

l5 specimen 2! which at low magni f icat ion 

v,hose im. 9 e is to be digit The low 

- -Vol nz^TZ^::^ m ^ TV:r 

in FIGS- l tnro 9 DOW er. a high magnification 

2 Ve~/™nstance-.t ,0X power and , control window 
rmage 26 for magnif icat ion image may have 

or image 28 The 1 S ^ reprclducad « 

» identified ther.i„ region » „ windo „ 

can «.iew architect a! region^of^nteres^^ ^ ^ 
megnification image 2, magni£ic at ion screen or 

3 0 bigh magni „„„„„ the cella forming a 

;:«::„ o £ the architectural feature need be examined 

fr,r rancer or the like or not. 
further fo 10 ie cons truct«d around a PC! 

„ 3 has a first Pentium Pro microprocessor 
» system bus ^"^ium £ „ icrop rocessor 4, connected 
4 2 and a second pentium P 

thereto. The system bus 40 has connected to 
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50 and an ISA bus 52. The PCI bus 50 has a SCSI 
controller 60 connected thereto to send and receive 
information from a hard disk 62. The hard disk 62 also 
is coupled in daisy chain SCSI fashion to a high capacity 
5 removal disk and to a CD Rom drive 66. The hard disks 62 
contains the programs for operating the system for 
controlling the microscope 16 and for processing the 
images as well as for doing a quantitative analysis of 
the selected portions of the histological specimens being 
10 viewed on the slide 18. The system bus 40 also has 

connected to it a random access memory 70 within which 
portions of the program being executed are stored as well 
as a read only memory 72 for holding a bootstrap loader 
as well as portions of the basic input/output operating 
15 system. A floppy disk controller 74 is coupled to the 

system bus 4 0 and has connected to it a floppy disk drive 
-76 for reading and writing information to a floppy disk 
f'as appropriate. A mouse controller 80 is coupled to the 
system bus and has a mouse 82 which operates as a 
20 pointing device for controlling manipulations on the 
screen 22 and within the windows 24 , 26 and 28. A 
keyboard controller 90 is connected to the system bus and 
has a keyboard 92 connected thereto. The keyboard 92 may 
be used to send and receive alpha numeric signals to 
25 other portions of the computer. An audio controller 100 
has a plurality of speakers 102 and a microphone 104 
connected thereto for audio input and output and is 
coupled to the system bus 40. A network interface, such 
as a network interface card 104, is connected to the 
30 system bus and can provide signals via a channel 106 to 
other portions of a network or internet to which the 
system may be connected. Likewise, signals can be sent 
out of the system through a modem 110 connected to the 
ISA bus 52 and may be sent via a channel 112, for 
35 instance, to the internet. A printer 116 is connected 
via a parallel I/O controller 118 to the system bus in 
order to provide printouts as appropriate of screens and 
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«„ it is generated. A serial I/O 
other information as bus and has 

controller 122 is ^ C ^°J* Br " 24 which is coupled 
connected to it a came The CCD sensors 126 

to CCD sensors 126 in the cam _ ative of wha t is 

S supply Pixel or image .xga.1 ^ P ^ acquisicion 

found on the slide 18 to ^ ^ 

controller 130 coupled to t ^ ^ ^ having . 

The microscope 16 xnc ^ objective 

stage 20 positioned ^ ^"/objectives 144. 146 and 
10 turret X42 having a * for inst ance. may be of 

14 . thereon. The l46 roay he a 20X 

- 25x objective. The ^ objectl ve. 

active. The »^ an d controller are 

Signals from the camera acquisition .y.w- 

supplied over a bus 128 to J- ^ the pcI bus for 

„here they are ^i.—ed and s PP ^ ^ disk 



15 



objective. The ^"""^J^. and controller are 
Signals from the camera acquisition system 

supplied over a bus 128 to t ^ the pcI bus fo, 

^ S "In T^Z£» -rage on the hard disK 

storage in IW 

62- on the slide 18 the stage 20 

When a specimen is on the CO mputer 



20 

controller 122. "^^^ such as the 
controls -pects of the ~ ^ or spectra l output of 

illumination the ins tance, in normal 

a lamp 168 and the like. ^ ^ slide 

operation, when a stage 20 in a step 

s P p ecimen slide IS is place - ^ ^ 42 or 44 s d . 
200, as shown m FIG. 14. ^ ^ serial I/c 

command through th «.^ C micros cope controller to 
controller 122 to «9»1 ^ & ^ 2Q2 . This is 

change magnification to 1.2 ^ ^ ^^.n 2 

done by rotating the object Likewise, the 

Microscope to select the ' of th e lamp 168. 

— ^ ^ ^ 

sets a pair of neutr correct 
illumination. ^ 
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controlled and a color filter wheel 180 may also be 
controlled to apply the appropriate filter color to the 
CCD censors 126 in the camera. The entire slide is then 
scanned in a step 204. The images are tiled and melded 
5 together into the overall image 24 supplied on the screen 
22 to provide the operator in the step 206 with a 
visually inspectable macro image of relevant regions of 
the slide of interest. 

In order to provide the magnified image, the 

10 mouse may be moved to identify a marker segment or region 
which, for instance, may be a rectangular region which 
will cause the microscope to change magnification as ac 
step 208 to 4x, 20x, 40x, etc., by rotating the turret to 
bring the appropriate objective lens system into viewing 

15 position . 

Next the user, in a step 209a, uses the mouse to 
select the region on the macro image in order to select 
the micro image to be viewed on the screen 22. In a step 
209b a test is made to determine whether the user has 

20 commanded continued inspection. If the user has, a test 
is made in a step 209c to determine if the magnification 
is to be changed by changing the selected objective. In 
the event the magnification is to be changed control is 
transferred to the step 208. If the magnification is to 

25 remain unchanged control is transferred to the step 209a. 
In the event inspection is not to continue the region 
selected is outlined for higher magnification scan in a 
step 209d. In a step 209e, a command may be received to 
scan or acquire the higher magnification image for 

30 display in screen 26. The image may then be archived for 
later analysis, displayed or analyzed immediately. 

In order to perform the magnification called for 
in step 20 8, the overall illumination and control of the 
microscope will be controlled so that in a step 210 the 

35 objective turret 142 will be rotated to place the higher 
power objective above the slide 18. In a step 212 
voltage to the lamp will be changed to adjust the lamp 
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position tne si 
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for each of the X and * ~ image is captured by 

coordinate values and the 
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the cameras and stored in RAM and backed up on the hard 
disk. The image may be then analyzed quantitatively in 
various manners such as those set forth in the 
previously-identified United States application. 
5 Optionally the image may be stored for archival purposes 
in a step 254 . 

In order to override the specific control 
functions that take place as shown in FIG. 12, a screen 
is provided as shown in FIG. 13 wherein the X-Y step size 
10 can be edited, the X, Y and Z offset can be edited, the 
lamp voltage can be selected, the neutral density filter 
can be selected as well as the opening of the field 
diaphragm and several other microscopic characteristics. 
FIG. 13. is a view of the settings of the microscope 
15 objective properties of the Axioplan 2, computer- 
controlled microscope . 

The X and Y positioning is specifically carried 
out as shown in FIG. 16 where the slide 18 is shown with 
a slide boundary 270, 272, 274 and 276. Stage boundary 
20 for limits of the stage travel for purposes of the stage 
the stage can be moved all the way from an upper left 
hand corner of travel 276 to a lower right hand corner of 
travel 280. At the upper left hand bounded corner of 
travel 278 limits which a signal that the end of travel 
25 has been reached and the stage is then translated a short 
distance 282 in the extra action and a short distance 284 
in the Y direction to define the first tile 288 in terms 
of a reference point 290 at its upper left hand corner. 
Since the size of the macro image tile 288 is known, the 
30 next macro image tile 292 may be placed contiguous with 

it by moving the stage appropriately and by measuring the 
location of the stage from the stage in counters without 
the necessity of performing any image manipulation. The 
image tiles 288 and 292 may be abutted without any 
3 5 substantial overlap or they may be overlapped slightly, 
such as a one pixel with overlap, which is negligible 
insofar as blurring of any adjacent edges of abutted 
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. The upper left hand corner 300 of the tile 
image tile. Th PP ^ ^ tiles can be so 

292 defines the rest or likewise be defined so 

s oven.pp.ns. .. encount ered with 

This avoids Che proo diait.l images in a 

!»»e storer or multiple without blurriness at 

or hring the images „ „ sy be 

10 the edges of contiguous image 1 _ plur ality 

, ppr e=iated ^^ETa^ ~ "* 
of micro images defined tn indiv idual tiles 

are shown in -^" h m Ti"e n addition, the region 3,0 
312 , 314, 316 and the ^ ^ may excee .d the 

15 when magnified *»°™Jus tie window may include 
bounds of the window and thus t . mage 3l0 

scroll bars or other means f « * examined from 

which is larger than the wxndow 26 

within the window 26 ^ ^ ^ and 

50 The sta9S , 79 and 281 with 

^ v stepper motors 279 ana 

includes the X and Y stePP ^ ^ loop 

th eir -spective encoder ; w ^ ^ g 

sys cem to «^^ cy 1 o " most mic roscope stages without a 
or 6 micron accuracy o ^ and thlS 

25 closed loop system T abuCting of the tile images 

ve ry high accuracy all ow unification images 

for both high mag-f and the 

without the currently used to eliminate the 

and expensive software overlappin g edges of 

30 overlap and blurry.. at^ positione d 
adjacent image tiles_ described in 

sta ge and » -« o ^^ ^ siide ±> preci 
connection with FIG . cp ^ the slxde , 

positioned relative to a can ^ taken frQm 

35 and the posiuon^ ^ precise ly in 

the same point, ~ . . _. _ n vrvuc ro 



a horizontal 



'tow and precised in vertid.i tows to 
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reconstruct the macro image and the micro image. This 
reconstruction is done without the use, as in the prior 
art, of extensive software manipulation to eliminate 
overlapping image tiles, horizontally or vertically or 
5 the haphazard orientation of image tiles. 

The present invention also includes the facility 
for allowing remote observation to occur by being able to 
couple the system either over a network communication 
facility to an intranet, for instance via the network 
10 interface, or via a modem or other suitable connection, 
to an internet so that once the image has been scanned 
and stored in memory on hard disks or other storage, 
remote users may be able to access the low magnification 
image as well as the high magnification image and move 
15 around within both images to make determinations as to 
the histological characteristics of the samples. 
> An additional feature of the system includes a 

plurality of networked workstations coupled to a first 
computer console 12 having a display screen 22 connected 
20 to the microscope 14. Satellite work stations 350 and 
352 are substantially identical to the work station 12 
including respective computers 354 and 356 coupled to 
displays 358 and 360. The devices can be manipulated 
through input devices 360 and 362 which may include a 
25 keyboard, mouse and the like. Also a third device can be 
connected including a work station 370, having a display 
372, a computer 374 and an input device 376. Each of the 
devices is connected over respective network lines 38 0, 
382, 384 to the computer 12 which transmission may be via 
30 either net or the like. Each of the different operators 
at the physically separate viewing stations can locate 
regions from the view of entire tissue cross sections via 
a macro view and label the regions for subsequent 
scanning and/or quantitative analysis. A single operator 
35 at the instrument station 12 can locate regions to view 
the entire tissue cross section. Those regions can be 
labeled for subsequent scanning and/or quantitative 
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vxzjzz ------ ::t".r ni rr° 

review analysis results while st ^ ^ 

„^ rh P entire macro view of the tissu 
5 reviewing the e ^i which the quantitative 

individual stored images fro «, tations 3 50, 352 and 

„ Kt . a ; ne d The viewing stations 
results were obtained. ar5toDS etc. There 

Hock top computers, laptops, et.^ 

rs°n c o an neirro: s : rcL^ - — < s — 3 50 . 

10 ^ 3 Z\ still further alternative embodiment 

ano 402 404, 406 and 408 may be 
remote workstations 400^ 402 ^ ^ ^ 

connected through a server 410 ^ a 

a pac.et switched «™f ' ser ver of the type 

15 hypertext transport protocol based ser 

^ fnr the World Wide Web or may be a telnet yv 

server 410 communicates via a 

applications. The ser computer 416 

communications channel 414 » » rh . r<suich the loe .l 
2 o havin, a display 418 »s»«. ^ „ pe 420 . Eac h 
computer 416 heing connected to the «e ^ 

o£ the r ;-™ ^n.' a'/the stations ISO, 3S3 
r„o r/althou^thev do it rrom " ^ 

25 £ rom around the world. ^J^^ ^^L. 

da "r o n stored La g es -y b. disseminated through the 

3 0 £ urther ^ysis^nd review^ ^ d „ cri . 

* «- v-^ nrpqent invention, it will 

35 intended in the appenae lowed in the true 

changes and modifications which followe 
spirit and scope of the present invention. 
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What la Claimed Is : 

1. A data structure of images taken from a 
specimen on a microscope slide comprising: 

a first series of contiguous, multiple images at 
a first magnification abutted against each other to 
create an overall low resolution view of several 
adjacent, original microscope images assembled together; 

the first series of multiple images taken from 
at least a portion of a specimen on the slide; 

a second series of contiguous, multiple images 
at a second higher magnification abutted against each 
other to create a high resolution view of several 
adjacent, original microscope images assembled together, 
taken from said portion of the specimen; and 
15 the first and second series of images providing 

multiple resolution images of the slide specimen to a 
viewer. 

2. A data structure in accordance with Claim 1 
wherein each of the series of contiguous, multiple images 

20 corresponds to an optical image seen by a person through 
an objective lens of a microscope. 

3. A data structure in accordance with Claim l 
wherein each of the series of contiguous, multiple images 
comprises a compressed image using a reduced percentage 

25 of a corresponding original microscope image. 

4 . A data structure in accordance with Claim l 
where i n an addressable coordinate system is provided for 
the first and second magnification images so that the 
higher magnification images can be easily located with 
30 respect to the lower magnification images. 



5. The data structure of Claim 4 wherein each 
of the image tiles in the first series and in the second 
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series inches »„ tai .»« .1— 

fhp entire image, 
reconstruction of the 

claim 4 further 
6 . The d ata struc tur o fQK viewing , 

comprising a dynamic, self to form 

S manipulating and reconstructing the 
the low and high resolution images. 

_ f pi R im 5 further 

7 TIT ZTZlllTt^ «~ — 

. v,-;nH resolution images . 
10 the low and high resoiu 

. The a,» -ua- of cl.i- - 
,« 1£ . ex .= utl n g program comprises . »v. 

„ * riaim 7 wherein said 
9 The data structure of Claim 

„ lf .„J t in 3 pro... ^prises » — -X .ppl.t. 

~ claim 7 wherein said 

THe data structure of Claim 

15 10 " Th mnrises a n internet web browser, 

self-executing program comprises an 

~ r^f claim 7 wherein said 
n The data structure of Claim 

self . eX ecut 1 ing program comprises an intranet browser. 

12 The data structure of Claim 7 further 

I- for scrolling through said first digital 

r/rr^irri: u — — 

13 The data structure of Claim 7 wherein said 
13 • fov displaying the coordinates of 

« P«*«« i - 1 ^%rr a s n t S irage an' sLd'second image, 
a point on said first image 

14 A storage medium having digitized images of 
. specimen on a microscope slide comprising: 
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a storage medium; 

a first series of contiguous, multiple images at 
a first magnification abutted against each other to 
create an overall low resolution view of several 
5 adjacent, original microscope images assembled together; 

the first series of multiple images taken from 
at least a portion of a specimen on the slide; 

a second series of contiguous, multiple images 
at a second higher magnification abutted against each 
10 other to create a high resolution view of several 

adjacent, original microscope images assembled together, 
taken from said portion of the specimen; and 

the first and second series of images providing 
multiple resolution images of the slide specimen to a 
15 viewer. 

15 . A storage medium in accordance with Claim 
14 wherein each of the series of contiguous, multiple 
images corresponds to an optical image seen by a person 
through an objective lens of a microscope. 

20 16. A storage medium in accordance with Claim 

14 wherein each of the series of contiguous, multiple 
images comprises a compressed image using a reduced 
percentage of a corresponding original microscope image. 

17. A storage medium in accordance with Claim 
25 14 wherein an addressable coordinate system is provided 

for the first and second magnification images so that the 
higher magnification images can be easily located with 
respect to the lower magnification images. 

18. The storage medium of Claim 14 wherein each 
30 of the image tiles in the first series and in the second 

series includes coordinate information for enabling 
reconstruction of the entire image. 
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x rlaim 14 further 
19 . The storage medium of Claim vie wing 
* am ic self-executing program for vi 
comprising a dynamic tile£ . 
manipulating and reconstructing 

^ rlaim 14 further 

~f rlaim 20 wherein said 
21 The storage medium of Claim 

inq program comprises a Java applet, 
self -executing proyo- 

~f rlaim 20 wherein said 
The storage medium of Claim z 

arises an Active-X applet. 
lf -executing program comprises 



viewing 



10 se 



15 



3 — 

~f rlaim 20 wherein said 
„. The .tor^-- H -" rnet web browser . 
self -executing program comprises 

^•f rlaim 20 wherein said 
2 « . The s» W n»diu» of weet 
, elf . e « Wt in g P,o 9 »» uprises an 

x: rlaim 20 further 

comprising means forSCr and a region on said 

irri age wherein select, Responding image at 

first digital image displays digital image, 

said second magnification xn said 

26 . The storage medium of Claim 
program includes means for 
a point on said first image and 

rlaim 14 wherein the 
„. The storage -ax- of ««- disks 
scorage ^ediun, is eexecced fro™ the g roup 



25 



and Jazz drive disks 



rhod of constructing a data structure 
28 . A method of slide comprising 

taken from a specimen on a mxcrosc p 



the steps of: 



* » 
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digitally scanning and storing a first series of 
digitized images taken from a portion of a specimen on a 
microscope slide in a first series of contiguous image 
tiles at a first magnification to allow formation of an 
5 overall view; 

digitally scanning and storing a second series 
of digitized images taken from the portion of the 
specimen on the microscope slide in a second series of 
contiguous image tiles at a second, higher magnification; 
10 and 

providing the data structure with the first and 
second series of digitized, stored images to provide a 
user with multiple resolution images from the specimen. 

29. The method of Claim 28 further comprising 
15 the step of: 

% providing the data structure with a dynamic, 

self -executing program effective for viewing, 
manipulating and reconstructing the image tiles. 

30. The method of Claim 28 further comprising 
20 the step of: 

providing each of the image tiles in the first 
series and in the second series with coordinate 
information for enabling reconstruction of the first and 
second images . 

25 31. The method of Claim 3 0 wherein said 

self -executing program comprises the step of providing a 
Java applet . 



30 



32. The method of Claim 30 further comprising 
the step of displaying the X-Y coordinates of a selected 
point on the first and second image. 
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w - f n a itn 28 further comprising 
33 The method of Claim 2B sto ring 

the digitized images and storing 
the steps of compressing the digi 
the compressed digitized images. 

, , nf claim 28 wherein the step of 
34 The method of Claim 

storing the images comprises: 

, taker, through Che microscope; and 

opticol outer P „ ti on S o£ t « 

image . 

v, ^ claim 28 wherein each of the 
35. The method of Claim ^ 
10 4= fhP series corresponds to 

digitized images of the a persQn through 

substantially an optical view 

an objective lens of the microscope. 

36 a storage medium having digitized images 
n a niece on a microscopic support 
15 from a specimen on a piece o 

comprising •• 

a storage medium; f of 

a first collection of <^iz ^ 
v,^firation coherently scoj-<=^ 
vie w ac a first •»9'""" tl ° overall low resolution view 

2 0 storage medium ZZZle images of the specimen on the 
from °"^nal mrcroscop ^ 

sli de "-" c » o t li a e / t l i r o S „ t o£ digitized image fields^ vrew 

,t a higher magnification ^"^^^^"t^image for 
25 storage medium to prov rde a hrg ^ ^ 

dewing from a selected th . fir6t an d 

. control program ored ^ ^ ^ . 

se cond °' nd d 9 o ch betwe en the overall view at 

^%r»rtTr and the selected higher 
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resolution images 



accordance with Claim 
« a storage medium in accord* 
37 . A scoiay ■ ale image fields 

«-k*> -First collection of singie 
36 wherein the first 
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comprises a collection of tiled images abutted one 
against the other. 

38. A storage medium in accordance with Claim 

37 wherein one tile is one picture viewed through the 
5 microscope. 

39. A storage medium in accordance with Claim 

38 wherein the second collection of image fields 
comprises at least three higher resolution image fields 
of view, each having a substantially different resolution 

10 and each capable of being selected by a user. 

40. A storage medium in accordance with Claim 

3 9 wherein the storage medium is on a web browser and the 
view is accessing the browser for viewing the specimen's 
digitized images. 

15 41. A storage medium in accordance with Claim 

40 wherein the stored, digitized images are JPEG images 
received over the Internet; and 

the stored control program comprises a HTML file 
and an active, self -executing program. 

20 42. A storage medium in accordance with Claim 

3 6 wherein the storage medium is a CD-Rom. 

43 . A storage medium in accordance with Claim 

41 wherein the active, self -executing program comprises a 
Java applet. 

25 44 . A method of constructing and using a self - 

executing data structure of an image of a specimen on a 
microscope slide comprising: 

digitally scanning and storing multiple 
magnification and multiple resolution images from the 

30 specimen on a microscope slide to create a plurality of 
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ln awidu.i i.ag. .la- ^-"iP" 

mulcipl e «-^;" a dyna „ i= , sel£ .e*ecu Cl ng pr^r.n. on 

th . dec. «ruc««« for viewing, manipulating and 
5 ^constructing Che image tiles, and 

.^ncferrinq the scanned, digital » 

10 multiple resolutions. 

_w ^ „f claim 44 further comprising 
45. The method of Ciaim 

the steps of: .„ m nriEina a portion of 

displaying a first image comprising a p 

. on as an overall macro view; and 
t h e specie a an uprising higher 

1S displaying imen on the microscope slid. 

overall macro view. 

^ ^ * riaitn 45 further comprising: 
46 The method of Claim *=> 

selecting a point on said overall macro image 
3 ^rrlducTng a corresponding higher magnification 
image at the location of the marker. 

47 . Th e method of Claim 44 further comprising 

X _Y coordinates of said point. 
25 displaying the X i coo* 

nf c i a i m 44 wherein said image 

48. The method ot txaim 

tiles are stored as bit-mapped files. 

49 The method of Claim 48 further comprising 
" . fn es to JPEG files, 

converting said bit-mapped files to 
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50. The method of Claim 44 , comprising scanning 
and storing said images of said specimen at at least 
three magnifications . 

51. A method in accordance with Claim 44 

5 including the step of transferring the digital images to 
a web browser. 

52. A method in accordance with Claim 51 
including the step of transferring of the digital images 
over a dynamic Internet. 

10 53 . A method in accordance with Claim 51 

including the step of transferring the digital images 
over a dynamic intranet. 

54. A method in accordance with Claim 51 
including the step of retoggling between lower 

15 magnification images and higher magnification images 
stored on the web browser. 

55. A method in accordance with Claim 46 
wherein the step of selecting a point includes: 

moving across tile boundaries to a desired point 
20 on a selected macro tile image; and 

executing a command to product the micro image 
from the selected macro tile image point . 

56 . A method in accordance with Claim 44 
including the step of: 

25 scrolling a portion of an image being viewed in 

a direction to cause the image being viewed to shift to 
include, in a new image, a portion of the image from a 
neighboring tiled image that was not previously viewed in 
the last image viewed by a user. 
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*nr creating a data structure of 
A program for creai^y 

• ^ n a microscope slide comprising: 

_ rr _ cnpciinen on a " ' x ^ ^ w ^ ^ 
an im age ^ tine digitally scanning the 

ima ge of the specimen on a microscope slide a, a 
5 P-^-^^-r— the scaled 

„ a series of contiguous image tiles; and 
digit .l S ;; u - ne for linking the series of 

Utiles with a dynamic, self -execute 
contiguous image tiles wit maniDU i a ting and 

offp . tive for viewing, manipulating 
10 program effective 

reconstructing the image tiles. 

„ f rlaim 57 further comprising 
co The program of Claim 

58. F a displaying a micro image 

• first display routine ^ V ^ ^ . first 

comprising a portion for displaying 

15 magnification and a seco over all view from the 

a macro second image comprising an over 
specimen on the microscope slide. 

™ i~>f claim 57 further comprising 
„ . The P^»» ° f C l a 2 said -aero image and 
a routine for selecting a point on s. 
2 0 for producing a corresponding micro rmage 

6 „ The program of Claim 57 further comprising 
a ooordinate display routine for displaying the 
coordinates of ..id point to the user. 

61 . » apparatus fcr creating a data structure 

25 <=°™^ iSin9 a co „ pute r-controlled microscope imaging system 
£ or digitally scanning images from . specimen on a 
horoscope support at a plurality of image 
magnifications, ^ ^ tmeBtiUv scamed 

30 ■„ a series of contiguous image tiles; and 

images in a series ^ ^ ^ ^ contiguous 

image tiles with a dynamic, self -executing program 
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effective for viewing, manipulating and reconstructing 
the image tiles. 

62. An apparatus is accordance with Claim 61 
wherein a data compressor compresses the data of the 

5 digitally scanned significantly to allow the scanned 
digitized images to be sent over the Internet. 

63. An apparatus in accordance with Claim 61 
wherein an addressable coordinate system provides 
addresses so that images can be seamed together and 

10 higher magnification images can be easily located with 
respect to the lower magnification images. 

64. An apparatus in accordance with Claim 61 
wherein a program for scrolling allows the user to scroll 
a portion of a neighboring image into view. 

15 65. An apparatus in accordance with Claim 61 

including an address display to display the coordinates 
to assist multiple viewers to identify the same area for 
analysis and commentary. 

66. An apparatus in accordance with Claim 61 
20 further comprising; 

a browser for storing the dynamic, self- 
executing program; and 

a monitor for viewing the images stored on the 
monitor and for flipping back and forth between low 
25 resolution macro images and high resolution micro images. 

67. An apparatus in accordance with Claim 66 
including a marker program to mark an addressable area on 
the macro image and to cause the addressed area to appear 
at a higher resolution, micro image on the monitor. 
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Fig. 7a 



Uncompressed Data Files in BMP format 
Contents of 'Dcis_027' 



FinalScan.ini 

SlideScan.ini 

DaO.bmp 

Da1 .bmp 

Da10.bmp 

Da100.bmp 

Da101 .bmp 

Da102.bmp 

Da103.bmp 

Da104.bmp 

Da1 1 bmp 

Da12.bmp 

Da13.bmp 

Da14.bmp 

Da15.bmp 

Da16.bmp 

Da17.bmp 

Da18.bmp 

Da19.bmp 

Da2.bmp 

Da20.bmp 

Da21 .bmp 

Da22.bmp 

Da23.bmp 

Da24.bmp 

Da25.bmp 

Da26.bmp 

Da27.bmp 

Da28.bmp 



Da29.bmp 

Da3.bmp 

Da30.bmp 

Da31 .bmp 

Da32.bmp 

Da33.bmp 

Da34.bmp 

Da35.bmp 

Da36.bmp 

Da37.bmp 

Da38.bmp 

Da39.bmp 

Da4.bmp 

Da40.bmp 

Da4l.bmp 

Da42.bmp 

Da43.bmp 

Da44.bmp 

Da45.bmp 

Da46.bmp 

Da47.bmp 

Da48.bmp 

Da49.bmp 

Oa5.bmp 

Da50.bmp 

Da51.bmp 

Da52.bmp 

Da53.bmp 

Da54.bmp 



Da55.bmp 

Da56.bmp 

Da57.bmp 

Da58.bmp 

Da59.bmp 

Da6.bmp 

Da60.bmp 

Da61 .bmp 

Da62.bmp 

Da63.bmp 

Da64.bmp 

Da65.bmp 

Da66.bmp 

Da67.bmp 

Da68.bmp 

Da69.bmp 

Da7.bmp 

Da70.bmp 

Da71.bmp 

Da72.bmp 

Da73.bmp 

Da74.bmp 

Da75.bmp 

Da76.bmp 

Da77.bmp 

Da78.bmp 

Da79.bmp 

Da8.bmp 

Da80.bmp 



Da81 .bmp 

Da82.bmp 

Da83.bmp 

Da84.bmp 

Da85.bmp 

Da86.bmp 

Da87.bmp 

Da88.bmp 

Da89.bmp 

Da9.bmp 

Da90.bmp 

Da91 .bmp 

Da92.bmp 

Da93.bmp 

Da94.bmp 

Da95.bmp 

Da96.bmp 

Da97.bmp 

Da98.bmp 

Da99.bmp 

Ss1 .bmp 

Ss10.bmp 

Ss1 1 .bmp 

Ss12.bmp 

Ss13.bmp 

Ss14.bmp 

Ss15.bmp 

Ss16.bmp 

Ss17.bmp 



Fig. 7b 

BliFinalScanFrame.class 
BliMessageBox.class 
BHWebSlide. class 



Ss18.bmp 

Ss19.bmp 

Ss2.bmp 

Ss20.bmp 

Ss21 .bmp 

Ss22.bmp 

Ss23.bmp 

Ss24.bmp 

Ss25.bmp 

Ss26.bmp 

Ss27.bmp 

Ss28.bmp 

Ss29.bmp 

Ss3.bmp 

Ss30.bmp 

Ss31.bmp 

Ss32.bmp 

Ss33.bmp 

Ss34.bmp 

Ss35.bmp 

Ss36.bmp 

Ss37.bmp 

Ss4.bmp 

Ss5.bmp 

Ss6.bmp 

Ss7.bmp 

Ss8.bmp 

Ss9.bmp 

mda027.TRA 
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lndex.HTML for file 'Deis 027' 



Contents of 'Deis 027' 



weDoiicie 


ua29.jpg 


Da55.jpg 


Daol .jpg 


Ss18.jpg 


index.html 


Da3.jpg 


Da56.jpg 
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Ss19.jpg 


DaO.jpg 


Da30.jpg 


Da57.jpg 


Da83.jpg 


Ss2.jpg 


Da1 .jpg 


Da31 .jpg 
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